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ABSTRACT 


The  patient  satisfaction  survey  (PSS)  of  the 
inpatient  area  at  National  Naval  Medical  Center  has  a 
low  patient  response  rate.  Historically,  less  than  six 
percent  of  the  patient  population  responds  to  the 
survey.  A  review  of  the  literature  documents  that  a 
low  response  rate  produces  a  biased  survey  which  will 
not  be  useful  for  management  decision  making  or  for  a 
legitimate  quality  assessment  program. 

The  purpose  of  this  paper  is  to  describe  a  study 
which  implements  a  staff  education  program  and  a 
standardized  methodology  for  conducting  the  survey  in 
an  attempt  to  increase  the  patient  response  rate.  An 
experimental  model  is  used  to  study  the  effects  of  the 
changes  among  the  involved  nursing  units.  A  chi-square 
test  was  used  to  test  the  difference  in  the  rate  of 
return  of  the  questionnaires  among  the  groups  studied. 

A  statistically  significant  difference  was  found 
between  the  group  who  received  training  and  method 
standardization  and  the  group  who  received  no 
treatment.  The  study  suggests  that  staff  training  and 
method  standardization  contribute  to  increasing  the 
response  rate  of  the  PSS. 
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I.  INTRODUCTION 


The  Joint  Commission  on  Accreditation  of 
Healthcare  Organizations  (JCAHO)  requires  a  hospital  to 
have  a  system  in  place  which  collects  information 
concerning  the  patients'  satisfaction  with  the  care 
being  provided  by  the  hospital  (1992) .  Parker  and 
Kroboth  (1991)  emphasized  the  need  for  an  adequate 
sample  to  be  present  to  provide  an  accurate  assessment 
of  the  surveyed  population. 

This  project  describes  an  attempt  to  identify  a 
method  to  improve  the  effectiveness  of  the  current 
survey  process  to  increase  the  response  rate  of  the 
patients  being  sampled.  As  health  care  organizations 
embrace  quality  improvement  theories,  the  dependence  of 
the  supplier  on  accurate  consumer  information  will 
increase  (Rubin,  1991) .  The  consumer  must  have 
adequate  representation  to  provide  meaningful 
information  for  decision  making  by  the  health  care 
organization. 

Background  of  the  study 

The  National  Naval  Medical  Center  (NNMC)  is  a  427 
bed  teaching  hospital  which  offers  multispeciality 
services  to  its  patient  population.  There  are 
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approximately  twelve  hundred  discharges  from  the 
inpatient  area  per  month  at  NNMC.  Historically,  the 
patient  satisfaction  survey  (PSS)  has  had  a  response 
rate  of  less  than  six  percent  of  the  patients  being 
discharged  from  the  inpatient  nursing  units. 

The  PSS  process  is  the  responsibility  of  the 
Patient  Relations  Department  (PRD)  which  has  direct 
accountability  to  the  Deputy  Commander  of  NNMC.  The 
results  of  the  PSS  are  reported  in  departmental  minutes 
to  the  Commander  through  the  quality  assessment 
monitoring  process. 

The  mandate  for  the  PSS  is  provided  in  two  NNMC 
instructions.  NNMC  Instruction  6010. 3B  provides  for 
the  PSS  results  to  be  monitored  as  part  of  the  quality 
assessment/ improvement  program.  NNMC  Instruction 
6320. 6A  places  the  responsibility  for  conducting  the 
PSS  under  the  patient  relations  program. 

Currently  the  PSS  is  being  conducted  by  the  PRD  on 
a  continuous  basis  throughout  all  inpatient  areas.  The 
PRD  uses  a  questionnaire  to  collect  information 
concerning  the  patients'  satisfaction.  The 
questionnaire  is  a  combination  of  close-ended  forced 
choice  and  open-ended  questions.  Content  validity  was 
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established  using  an  expert  committee  review  process  at 
the  time  the  questionnaire  was  created. 

The  degree  to  which  an  instrument  has  the  ability 
to  measure  the  characteristic  that  is  being  studied  is 
referred  to  as  validity.  Reliability  refers  to  the 
instrument's  ability  to  consistently  measure  the 
characteristic  or  concept  being  studied  (Burns  &  Grove, 
1987) .  An  instrument  must  be  reliable  to  be  valid, 
however,  reliability  alone  does  not  indicate  that  an 
instrument  is  valid.  The  instrument  must  consistently 
measure  the  desired  characteristic  in  order  to  be  both 
reliable  and  valid  (Polit  &  Hungler,  1985) .  A  reliable 
and  valid  patient  satisfaction  questionnaire  would 
consistently  measure  those  characteristics  which  have 
been  found  to  make  patients  satisfied  with  the  care 
received  at  NNMC. 

The  PRD  prepares  the  PSS  questionnaire  to  be  used 
in  the  inpatient  areas  and  distributes  the 
questionnaire  to  the  Department  Heads  of  nursing 
service.  The  department  heads  then  circulate  the  PSS 
to  the  various  division  officers  in  the  department. 

The  division  officers  are  tasked  with  conducting  the 
surveys  at  the  nursing  unit  level.  In  some  cases  the 


4 


questionnaire  may  be  given  directly  to  the  patient  by 
the  PRD  staff  during  a  patient  visit.  No  standard 
procedure  ensures  the  proper  distribution  of  the 
questionnaire  to  the  patient  or  determines  at  what 
point  in  the  hospitalization  the  questionnaire  is  given 
to  the  patient.  The  patient  is  instructed  to  drop  the 
completed  form  into  the  suggestion  box  located  at  each 
elevator.  The  elevators  are  located  off  the  nursing 
unit,  and  the  suggestion  boxes  are  not  always  clearly 
identified  for  the  patient. 

The  current  method  of  data  collection  is  flawed 
for  the  following  reasons:  (1)  there  is  no  formal 
training  program  for  the  staff  of  the  nursing  unit 
concerning  the  importance  of  the  PSS,  (2)  there  is  no 
emphasis  on  the  importance  of  staff  cooperation  in  the 
data  collection  process,  and  (3)  there  is  no  uniform 
method  used  to  collect  the  information.  The  training 
provided  to  the  staff  of  the  nursing  unit  is  dependent 
upon  the  value  that  the  PSS  has  to  the  division 
officer.  Frequently,  the  PSS  questionnaire  is 
distributed  on  the  nursing  unit  only  if  the  nurse  feels 
that  there  is  extra  time  to  spend  with  the  patient,  or 
the  division  officer  has  emphasized  the  importance  of 
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the  PSS  to  the  staff.  Information  concerning  data 
collection  and  the  survey  process  is  provided  from  the 
department  head  level  on  an  infrequent  basis. 

PRD  does  not  provide  feedback  to  the  nursing  unit 
concerning  the  patient  response  rate  to  the  PSS.  A 
final  analysis  is  completed  by  PRD  based  on  the  total 
number  of  completed  questionnaires  received  from  each 
unit.  No  attempt  to  identify  a  rate  of  return  by 
relating  the  response  rate  of  the  questionnaires  to  the 
number  of  discharges  on  each  unit  is  made. 

Statement  of  the  Research  Question 

Will  providing  training  to  the  nursing  unit  staff 
and  standardizing  the  survey  process  increase  the 
inpatient  response  rate  to  the  PSS  at  NNMC? 

Review  of  the  Literature 

The  introduction  of  the  Total  Quality  Management 
(TQM)  philosophy  into  the  health  care  industry  has 
increased  the  importance  of  listening  to  the  customer 
(Matthews,  1992) .  Health  care  organizations  must  have 
access  to  client  information  to  have  a  successful 
quality  improvement  program  (Orme,  Parsons,  and 
McBride,  1992;  Bausell,  1985).  El-Guebaly,  Toews, 
Leckie,  and  Harper  (1983)  assert  that  using  the  patient 
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as  a  source  of  feedback  is  important  to  ensure  the 
congruence  of  the  provider/client  objectives  in  the 
provision  of  health  care.  A  system  which  can  be 
sensitive  to  identifying  the  patients'  needs  in  an 
economical  way  has  been  difficult  to  develop  (Attkisson 
&  Zwick,  1982) . 

The  determination  of  patient  satisfaction  is 
important  for  the  successful  recovery  of  the  patient 
once  discharged  from  the  hospital.  Positive  patient 
satisfaction  with  inpatient  care  has  been  related  to 
the  predictability  of  compliance  with  treatment  after 
leaving  the  hospital  (Rubin,  1990;  Baker,  1983) .  The 
PSS  needs  to  accurately  identify  problems  which  the 
patient  experiences  while  receiving  care  (Ware,  1981) . 

A  PSS  with  a  low  response  rate  limits  the 
usefulness  of  the  information  obtained  on  the 
questionnaire.  Kotsopoulos,  Elwood,  and  Oke  (1989) ; 
Rubin  (1990)  found  that  a  low  response  on  the  PSS  will 
distort  the  findings  since  patients  who  are  satisfied 
are  more  likely  to  respond  to  a  survey.  Patients  who 
are  nonrespondents  are  more  apt  to  be  dissatisfied  with 
the  care  received  than  the  respondents  to  a  PSS.  The 
proportion  of  respondents  to  nonrespondents  should  be 
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considered  when  evaluating  the  PSS  results.  To  be  of 
value,  a  PSS  must  achieve  a  response  rate  which 
accurately  reflects  the  population  being  served 
(Parker  &  Kroboth,  1991) . 

There  are  no  set  proportions  for  the  size  of  the 
sample  that  will  accurately  reflect  the  population 
being  studied.  In  general,  the  larger  the  sample  size, 
the  more  likely  that  sample  will  represent  the 
population  being  studied.  Woods  and  Catanzaro  (1988) 
report  that  there  are  three  factors  that  influence 
sample  size:  (l)  amount  of  variance  of  the  phenomenon, 
(2)  the  statistical  test  being  used,  and  (3)  effect 
size. 

Highly  variable  phenomena,  such  as  health  care 
values,  require  a  higher  proportion  of  the  population 
to  be  sampled  (Woods  &  Catanzaro,  1988)  in  order  to 
have  accurate  measurements.  Conversely,  phenomena  with 
low  variability  can  be  represented  with  a  smaller 
sample  of  the  population  being  studied. 

The  sample  size  needed  is  affected  by  the 
statistical  test  being  used  by  the  researcher  (Woods  & 
Catanzaro,  1988) .  A  sample  size  must  be  large  enough 
to  reach  a  level  of  significance  that  decreases  the 


8 


risk  of  rejecting  the  null  hypothesis  as  a  result  of 
error . 

The  effect  size  refers  to  the  magnitude  of  the 
finding  or  the  level  of  rejection  of  the  null 
hypothesis  (Woods  &  Catanzaro,  1988) .  The  greater  the 
effect  size  is  on  the  population  being  studied,  the 
smaller  the  sample  can  be  to  obtain  meaningful  results. 

The  timing  of  the  survey  is  important  in  achieving 
an  adequate  response  rate.  Rubin  (1990)  reports  that 
the  longer  the  time  period  between  the  medical 
treatment  and  the  PSS,  the  less  likely  the  patient  will 
respond  to  a  survey.  Those  patients  sampled  while 
still  in  the  hospital  or  several  months  after  discharge 
had  a  higher  satisfaction  rating  than  those  surveyed 
shortly  after  discharge.  The  advantage  of  conducting 
surveys  at  the  time  of  discharge  is  that  the  patient 
has  gained  an  overview  focus  of  treatment  and  is  still 
in  the  facility  (Lebow,  1982).  A  possible  disadvantage 
to  a  PSS  conducted  at  time  of  discharge  is  that  any 
negative  information  obtained  cannot  be  used  to  change 
the  system  to  a  more  positive  image  while  the  patient 
is  still  in  the  hospital  (Weiss  &  Senf ,  1990) .  To 
obtain  the  needed  information,  the  data  collection 
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system  for  the  PSS  has  to  be  convenient  for  both  the 
staff  and  the  patient  (Parker  &  Kroboth,  1991) .  The 
effective  use  of  resources  for  distribution  and 
collection  of  the  PSS  may  facilitate  an  increasing 
response  rate  to  the  questionnaire  (Ware,  1981) . 

Training  is  the  element  of  organizational  support 
frequently  missing  in  programs  which  require 
participation  of  the  staff  (McGraw,  1992) .  Walton 
(1986) ,  in  writing  about  TQM,  reports  that  training  at 
the  user  level  is  vital  to  the  success  of  any  program 
requiring  user  participation.  Studies  have  identified 
the  importance  of  education  in  gaining  both  compliance 
with  and  support  of  programs  at  both  the  staff  and  the 
patient  level  (Bird,  1992;  Gariti,  Greenstein,  Olsen,  & 
Harris,  1987;  Jones,  Jones,  &  Katz,  1988).  An 
effective  training  program  should  increase  the  value  of 
the  PSS  to  the  staff,  and  therefore  result  in  a  higher 
level  of  staff  participation  and  a  greater  response 
rate  to  the  survey  (Leuze,  1990). 

The  involvement  of  the  staff  in  quality  assessment 
programs,  such  as  PSS,  enhances  the  effectiveness  of 
the  program  (Mulcahy  &  Wagner,  1991) .  Marketing  the 
survey  as  a  means  to  identify  the  contribution  which 
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the  nursing  staff  makes  to  successful  patient  care 
increases  the  likelihood  of  their  participation  in  the 
survey  process  ( Sawyer-Richards ,  1990) .  The  PSS  can 
then  be  viewed  as  part  of  patient  care  versus 
additional  work  for  the  staff.  A  transition  must  be 
made  to  shift  the  responsibility  of  the  PSS  from  a  one 
person  or  committee  function  to  an  all  staff 
involvement  mentality  (Bevsek  &  Walters,  1990) . 

Purpose  of  the  Study 

The  purpose  of  this  study  is  to  identify  variables 
which  will  increase  the  rate  of  response  on  the  PSS 
questionnaire  at  NNMC.  The  null  hypothesis  for  this 
study  is; 

H0:  There  is  no  difference  in  the  PSS  survey  response 
rate  of  patients  in  the  training  and  method 
standardization  group,  the  training  only  group,  the 
method  standardization  only  group,  and  the  no  treatment 
group  (control) . 

The  alternate  hypothesis  for  this  study  can  be  stated 
as; 

Hj^:  There  is  a  difference  in  the  PSS  survey  response 
rate  of  patients  in  the  training  and  method 
standardization  group,  the  training  only  group,  the 
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method  standardization  only  group,  and  the  no  treatment 
group  (control) . 

STUDY  DESIGN 

The  approach  to  this  study  is  the  experimental 
model.  Beach  (1992)  describes  the  experimental  method 
as  a  way  to  study  the  relationship  of  manipulated 
variables  with  the  use  of  a  control  group. 

The  study  involved  eight  nursing  units  of  the 
inpatient  area  at  NNMC.  The  eight  nursing  units  were 
divided  into  four  sections:  (1)  a  no  treatment  unit, 

(2)  a  training  only  unit,  (3)  a  method  standardization 
only  unit,  or  (4)  a  training  and  method  standardization 
unit  (see  Appendix  A) .  A  nonrandomized  convenience 
method  of  selection  (Polit  &  Hungler,  1985)  was 
utilized  for  determining  which  nursing  units 
participated  in  the  study.  Convenience  sample 
methodology  is  the  selection  of  participants  merely  due 
to  location,  timing,  or  a  need  for  a  certain  number  of 
participants.  The  eight  units  selected  for  this  study 
are  located  close  to  each  other  and  have  a  high  patient 
turnover.  The  close  proximity  of  the  units  and  the 
high  patient  turnover  provided  for  easier  management  of 
the  project  and  a  potentially  large  sample  size. 
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The  training  was  conducted  prior  to  data 
collection  and  attempted  to  reach  eighty  percent  of  the 
staff  on  the  participating  nursing  unit.  An  eighty 
percent  level  of  staff  participation  was  identified  due 
to  staff  absences  resulting  from  training  requirements, 
illness,  and  annual  leave. 

The  content  of  the  class  included  the  thrt 
elements  that  Sawyer-Richards  (1990)  identified  as 
producing  the  desired  outcome  for  increasing  staff 
support  and  participation  (see  Appendix  B) : 

1.  Professional  accountability 

2.  Feedback  on  the  usefulness  of  the  activity 

3.  Acknowledgment  of  the  improved  results  with 
staff  participation. 

The  method  standardization  was  designed  to 
facilitate  the  participation  of  the  staff  and  to 
provide  for  the  convenience  of  the  patient.  The 
questionnaire  was  placed  on  each  patient  chart  with 
other  forms  utilized  during  the  discharge  phase  of  the 
patients'  hospitalization.  The  nurse  reviewed  the 
questionnaire  with  the  patient  during  discharge 
teaching  prior  to  the  patient  leaving  the  unit.  The 
patient  was  instructed  to  deposit  the  PSS  questionnaire 
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in  collection  boxes  placed  near  the  nurses  station  as 
they  leave  the  unit  (for  greater  detail  see  Appendix 
C) .  The  design  goal  was  to  increase  the  response  rate 
of  the  questionnaire  through  increasing  awareness  and 
convenience  to  the  patient  and  staff. 

Subjects 

The  participants  of  this  study  include  both  the 
staff  and  the  patients  of  NNMC.  The  Registered  Nurses 
include  active  duty  commissioned  officers,  civil 
service,  and  contract  staff.  The  nonregistered  nursing 
staff  is  the  enlisted  hospital  corps  staff  of  E-5s  or 
below. 

Patients  at  NNMC  cover  a  wide  range  of 
beneficiaries  including  the  following: 

-  Active  Duty 

-  Retired  Military 

-  Officer 

-  Enlisted 

-  Dependents 

-  Male  and  Females 

-  All  age  groups 

-  Secretary  of  the  Navy  Designates 

Due  to  the  nature  of  this  study  and  for  the  protection 
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of  human  subjects,  no  attempt  was  made  to  identify 
individual  participants  in  any  way.  Statistics  were 
compiled  to  describe  group  relationships  and  to 
determine  whether  there  were  any  difference  among  the 
groups . 

Treatment  of  the  Data 

The  acceptance  or  rejection  of  the  stated 
hypothesis  is  based  on  the  statistical  analysis  of  the 
four  groups.  The  four  groups  under  consideration  in 
this  study  are  the  training  and  method  standardization 
group,  the  training  only  group,  the  method 
standardization  only  group,  and  the  no  treatment  group. 
Hunt  (1982)  describes  the  chi-square  statistical  test 
as  one  of  the  most  flexible  techniques  available  for 
decision  makers  use.  Chi-square  is  particularly  useful 
with  nominal  data  (Cohen,  1988) .  The  number  of 
responses  observed  in  comparison  to  the  number  expected 
was  used  in  the  data  analysis.  The  number  of  surveys 
returned  versus  the  number  of  discharges  for  each 
nursing  unit  was  monitored  during  the  six  weeks  of  the 
project.  At  the  end  of  a  six  week  period,  the  total 
number  of  surveys  returned  weekly  for  each  of  the 
groups  was  compared. 
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The  response  rate  of  the  treatment  groups  was 
compared  to  identify  whether  a  significant  statistical 
relationship  exists.  The  chi-square  statistic  was  used 
to  test  the  null  hypothesis  for  this  study. 

FIMDING8 

During  the  six  week  period,  a  total  of  105  PSS's 
were  collected.  Table  1  details  the  number  of 
respondents  by  gender.  Active  duty  and 
retired  individuals  were  the  largest  group  of 
respondents  (see  table  2) .  The  PSS  responses 
represented  patients  from  less  than  one  year  old 
(parent  or  caretaker  completing)  to  patients  greater 
than  eighty  years  old.  Forty  percent  of  the 
participants  were  between  the  ages  of  twenty-one  and 
fifty  years  of  age  (see  table  3) . 

Table  1 

Respondent  by  Gender 


Gender 

Number  of 
Responses 

Percent  of 
Response 

Male 

55 

52 

Female 

37 

35 

Not  Identified 

13 

13 

Total 

105 

100 

Status 

Number  of 
Responses 

Percent  of 
Responses 

Active  Duty 

34 

33 

Active  Duty 
Dependent 

18 

17 

Retired 

30 

29 

Other* 

23 

21 

Total 

105 

100 

♦includes  dependents  of  retirees,  Secretary  of  Navy 
designates,  etc. 


Table  3 

e  of  Respondents 


Age  in  Years 


Number  of 
Responses 


Percent  of 
Responses 


0-10 

(Adult 

Representative) 


11-20 


21-30 


31-40 


41-50 


51-60 


61-70 


71-80 


81  and  above 


Not  Identified 


Total 
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There  were  1064  discharges  from  the  eight  nursing 
units  participating  in  the  study  during  the  six  week 
period  with  a  total  of  ten  percent  of  those  patients 
discharged  completing  the  PSS.  The  nursing  units 
which  combined  treatments  of  training  and  method 
standardization  demonstrated  the  highest  percentage  of 
return  among  the  four  groups  of  nursing  units  in  the 
study.  Sixteen  percent  of  the  patients  discharged  from 
the  training  and  method  standardization  nursing  units 
responded  to  the  survey  (see  table  4)  compared  to  one 
percent  of  the  no  treatment  nursing  units. 

Table  4 

Response  Rate  bv  Groups 


Treatment 

Number  of 

Number  of 

Percent  of 

Responses 

Discharges 

Responses 

Training  and 
Method 

Standardization 

48 

297 

16 

Training 

20 

227 

9 

Method 

Standardization 

34 

273 

13 

No  Treatment 

3 

267 

1 

Overall 

Response  Rate 

105 

1064 

10 
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Based  on  the  results  of  the  chi  square  test,  the 
null  hypothesis  for  this  study  is  rejected  and  the 
alternate  hypothesis  is  accepted  X2(3,N=104)  =  38.518= 
E<  4.26  X  10"7.  Table  5  displays  the  chi-square  table 
with  row  and  column  values  for  each  of  the  groups 
participating  in  this  study. 

Table  5 

Chi-Scruare  Test  of  Variables 


Treatment 

No 

Response 

Response 

Total 

Training  and 

249* 

48 

297 

Method 

Standardization 

23.40“ 

4.51 

27.91 

Training 

207 

20 

227 

19.45 

1.88 

21.33 

Method 

239 

34 

273 

Standardization 

22.46 

3.20 

25.66 

No  Treatment 

264 

3 

267 

24.81 

.28 

25.09 

Total 

959 

105 

1064 

90.13 

9.87 

100.0 

Number  of  responses 

‘‘Percent  of  total  response  (may  not  add  up  due  to 
rounding) 


Discussion 


Previous  studies  (Nelson,  Rubin,  Hays,  &  Meterko, 


1990)  reported  that  females  are  more  likely  to  respond 
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to  PSS.  Sixty-one  percent  of  the  patients  on  the 
nursing  units  were  male  during  the  study.  Of  the 
eighty-seven  percent  who  identified  their  gender  fifty- 
two  percent  were  males  and  thirty-five  percent  were 
females.  Thirteen  percent  of  the  respondents  did  not 
identify  gender  on  the  questionnaire. 

The  patient  population  at  NNMC  contains  active 
duty,  retired  personnel  and  their  dependents.  Nelson  et 
al.  (1990)  reported  that  younger  patients  (age  <50 
years)  were  more  likely  to  respond  to  a  PSS.  The 
largest  response  rates  occurred  in  the  age  groups  of 
thirty-one  to  forty  (sixteen  percent  of  responses)  and 
sixty-one  to  seventy  (nineteen  percent  of  responses) . 

A  review  of  inpatient  admissions  at  NNMC  during  the 
study  reveals  that  sixty-four  percent  of  the  patients 
were  below  the  age  of  fifty  and  thirty-six  percent  were 
above  the  age  of  fifty.  Eleven  percent  of  the  patients 
did  not  supply  information  concerning  age.  Twenty-nine 
percent  of  the  sample  was  reported  to  be  retired 
military  while  fifty  percent  of  the  sample  was  reported 
to  be  active  duty  or  active  duty  dependents,  while 
twenty-one  percent  represented  "other"  classification 
(see  table  two) .  This  mixture  of  responses  of  both  age 
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and  status  reflects  both  the  active  duty  and  retired 
population  which  NNMC  serves. 

The  increased  rate  of  response  to  the  PSS  on  units 
with  both  training  and  method  standardization  is 
supported  by  previous  research  (Bird,  1992;  Gariti, 
et.al.,  1987;  Jones,  et.  al.,  1988:  Walton,  1986). 
Nelson  and  Niederberger  (1990)  reported  a  HMO  with  a 
response  rate  of  greater  than  seventy  percent  using  a 
combination  of  onsite  PSS  and  trained  surveyors.  The 
HMO  was  compared  to  other  institutions  which  used 
onsite  PSS  and  untrained  surveyors  and  reported  a 
response  rate  of  less  than  twenty  percent.  The 
effective  use  of  resources  combined  with  the  gaining  of 
organizational  support  through  training  resulted  in  a 
predictably  greater  response  to  the  PSS  in  this  study 
(McGraw, 1992 ;  Ware,  1981). 

A  factor  which  might  account  for  the  higher 
response  rate  on  the  units  being  studied  is  the 
Hawthorne  effect.  The  Hawthorne  effect  is  described  as 
a  reaction  occurring  when  research  subjects  change 
their  behavior  because  of  the  study  and  not  because  of 
the  treatment  (Burns  &  Grove,  1987) .  The  Hawthorne 
effect  may  occur  due  to  the  participants  wanting  to 
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please  the  researchers  or  trying  to  guess  the  outcome 
of  the  study  (Woods  &  Catanzaro,  1988) . 

The  frequent  interaction  of  the  researcher  in 
providing  feedback  with  method  standardization  may 
account  for  the  higher  percentage  of  responses  on  those 
units.  The  effect  of  perceived  higher  visibility  in 
the  command  by  the  subjects  may  also  account  for  a 
higher  percentage  of  return  of  the  questionnaire  on  the 
method  standardization  units. 

The  response  rate  to  the  over? 11  PSS  during  the 
study  increased  from  six  on  previous  surveys  to  ten 
percent.  Gallagher  (1989)  questions  the  validity  of  a 
survey  with  low  response  rate  since  there  is  a 
significant  number  of  people  who  have  not  responded 
(ninety  percent  in  this  study) .  The  danger  of 
overreporting  or  underreporting  a  narrow  spectrum  of 
patient  opinion  exists  (Gallagher,  1989) .  The  factor 
of  non-response  bias  as  reported  by  Rubin  (1990) 
creates  further  questions  concerning  the  validity  of  a 
PSS  with  a  ten  percent  return  rate.  Baker  (1983) 
reported  that  many  of  the  non-respondents  may  be 
dissatisfied  with  the  care  received,  and  therefore, 
important  information  may  go  unreported. 


22 


Recomaenda t ions 

A  PSS  system  which  will  increase  the  response  rate 
at  NNMC  is  needed.  Nelson  et.  al.  (1990)  identified 
the  use  of  a  telephone  survey  as  a  method  which  can 
provide  a  high  response  rate  to  the  PSS.  Nelson  and 
Niederberger  (1990)  reported  a  nonresponse  rate  of  less 
than  twenty  percent  when  a  telephone  survey  was  used. 
This  system  could  be  utilized  at  NNMC  with  additional 
investment  of  resources  on  the  part  of  the  patient 
relations  department. 

A  list  of  recently  discharged  patients  would  be 
forwarded  to  the  PRD.  The  PRD  representative  would 
then  call  the  patient  and  ask  the  questions  contained 
on  the  current  PSS  questionnaire.  As  the  questionnaire 
is  being  answered,  the  individual  conducting  the  survey 
would  encode  the  responses  on  the  PSS  automated  data 
base.  The  automated  data  base  is  already  in  place  with 
the  current  PSS  system. 

Staffing  requirements  for  the  PRD  may  increase  one 
additional  person  to  manage  the  program.  In  the  change 
of  PSS  methodology,  the  need  to  mass  print  the 
questionnaires  would  be  eliminated,  time  would  be  saved 
in  distributing  and  collecting  the  questionnaires,  more 
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efficient  use  of  personnel  would  result,  and  collection 
of  more  useful  information  would  be  accomplished.  Any 
cost  savings  or  increase  in  efficiency  which  results 
from  the  new  system  can  be  used  to  subsidize  the 
additional  manpower  requirement  of  the  PSS  system. 

An  additional  advantage  of  a  centralized  telephone 
PSS  system  includes  the  continuity  of  the  process. 

The  frequent  changing  of  nursing  unit  staff  requires 
frequent  training  of  personnel  concerning  the  PSS  . 

Less  impact  of  nursing  unit  staff  changes  would  be  felt 
on  a  centralized  PSS  system.  The  process  would  be 
independent  of  the  nursing  unit  staffing  levels 
providing  for  a  more  consistent  level  of  rate  of  return 
for  the  PSS. 

A  mail-out  system  of  PSS  could  be  used  to  increase 
the  effectiveness  of  the  phone  system.  The  mail-out 
system  would  be  used  to  compensate  for  the  portion  of 
the  population  who  may  not  have  telephones,  may  provide 
wrong  telephone  numbers,  may  have  a  physical  impairment 
which  impedes  the  use  of  a  telephone  or  who  wants 
greater  anonymity  than  is  provided  in  a  telephone 
survey  (Press  &  Ganey,  1989) . 

The  impact  of  the  nursing  unit's  opinion  leader 
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was  not  a  consideration  of  this  study.  Opinion  leaders 
are  those  individuals  that  exert  a  significant  amount 
of  social  influence  over  other  members  of  a  group 
(Myers  &  Robertson,  1972) .  Seto,  Ching,  Yuan,  Chu,  and 
Seto,  (1990)  found  that  the  success  of  a  program  on  the 
nursing  unit  can  be  impacted  by  the  opinion  leader. 
Further  studies  should  look  at  the  impact  of  the 
opinion  leader  in  the  relation  to  the  response  rate  of 
the  PSS. 

The  generalizability  of  this  study  is  limited  to 
the  population  represented  in  this  project.  The 
content  validity  of  the  current  PSS  questionnaire  used 
at  NNMC  should  be  reviewed  once  the  response  rate  is 
increased.  The  check  of  content  validity  would  ensure 
the  questionnaire  is  collecting  the  desired 
information. 

Conclusion 

The  PSS  is  important  in  maintaining  and  enhancing 
the  institution's  status  within  the  community  served 
(Swan,  Sawyer,  Van  Maire,  and  McGee,  1985) .  The 
purpose  of  this  study  was  to  identify  a  method  which 
will  increase  the  response  rate  to  the  PSS.  An 
increased  response  rate  is  needed  to  gain  a  more 
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accurate  picture  of  the  population  being  served.  This 
study  found  that  within  this  sample,  training  and 
method  standardization  made  a  significant  difference  in 
the  response  rate  of  the  PSS. 

This  study  identified  important  shortcomings  of 
the  current  inpatient  PSS  system  at  NNMC.  The  rate  of 
return  for  the  PSS  was  not  being  considered  when 
reporting  the  results  of  the  PSS.  The  results  of  the 
current  PSS  system  cannot  be  assumed  to  represent  the 
opinions  of  inpatients  at  NNMC.  The  current  PSS  system 
at  NNMC  needs  to  be  improved  in  order  to  gain  a  higher 
response  rate  that  is  more  representative  of  the 
patient  population. 

As  the  twentieth  century  comes  to  a  close,  health 
care  organizations  find  themselves  scrambling  to  stay 
alive.  Health  care  organizations  can  no  longer 
egocentrically  offer  a  service  with  the  expectation 
that  clients  will  use  the  service  without  regard  to 
need,  satisfaction,  or  quality.  The  organization  must 
be  proactive  in  identifying  client  needs  and  client 
problems  with  the  system.  Those  health  care 
organizations  which  effectively  use  a  PSS  system  will 
not  just  be  surviving,  but  will  be  leading  their  peers 
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into  the  twenty-first  century. 
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Appendix  A 


Participation  of  fcfae  Nursing  Units 


UNIT 

NO 

TREATMENT 

TRAINING 

ONLY 

METHOD 

ONLY 

TRAINING 

AND 

METHOD 

5W 

Medicine 

X 

3W 

Cardiology 

C-T*  Surgery 

X 

6E 

Neurology 

Neurosurgery 

X 

6W 

Oncology 

X 

4W 

Plastic 

Surgery 

Orthopedics 

X 

7E 

Pediatrics 

X 

5E 

Urology 

General 

Surgery 

X 

5C 

Oral  Surgery 
Gynecology 

X 

*Cardiothoracic  surgery 


Appendix  B 


Class  Outline  for  Unit  Training 
Length:  7-10  minutes 

Time:  To  be  Conducted  During  Shift  Change  Report 
Location:  Unit  Report  Room 
Content : 

1.  Importance  Of  Patient  Satisfaction  Survey 

A.  Survey  Mandated  by  JCAHO 

B.  Hospital  Commander  wants  to  know  how  patients 
feel  about  the  care  they  are  receiving. 

C.  Helps  to  identify  system  problems. 

D.  Helps  to  identify  the  good  things  that  are 
being  done  for  the  patient. 

E.  There  is  a  need  for  all  staff  members  to 
encourage  the  patients  to  complete  the 
questionnaires . 

2.  If  the  unit  was  participating  in  the  method  change: 

A.  Review  of  the  method  standardization  process 


Appendix  C 

Method  Standardization  for  the  PSS 
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1.  Methodology  for  the  PSS 

A.  The  researcher  meets  with  the  Division  Officer 
of  the  participating  unit  and  explains 
procedure . 

B.  The  PRD  representative  brings  the 
questionnaires  and  collection  box  to  the  unit 
and  reviews  the  procedure  with  the  senior 
corpsman. 

C.  The  PSS  questionnaire  will  be  placed  on  the 
patient  chart  after  admission  with  all  forms 
that  the  Nurse  reviews  with  the  patient  at  time 
of  discharge. 

D.  The  nurse  takes  the  questionnaire  to  the 
patient  and  gives  it  to  the  patient  as  part  of 
discharge  teaching. 

E.  The  patient  is  instructed  to  drop  off  the 
completed  questionnaire  at  the  nurses  station 
prior  to  leaving  the  unit. 

F.  The  patient  receives  a  second  reminder  to  drop 
off  the  questionnaire  when  checking  out  at  the 
nurses  station  prior  to  leaving  the  unit. 

G.  The  PRD  representative  will  collect  the 
completed  questionnaires  from  the  nursing  units 
every  Friday  for  the  previous  week  and  document 
how  many  are  from  each  unit. 

H.  The  researcher  will  than  calculate  the  percent 
of  completed  questionnaires  to  the  number  of 
discharges  for  the  unit. 

I.  The  nursing  units  which  received  training  will 
get  feedback  concerning  the  response  rate  of 
the  PSS  every  two  weeks  during  the  study. 


J.  At  the  end  of  the  six  week  period  the  data 
will  be  used  for  statistical  calculations. 


